Predicting and Interpolating State-level Polls using Twitter Textual Data

Spatially or temporally dense polling remains both difficult and expensive using existing survey methods. In response, there have been increasing efforts to approximate various survey measures using social media, but most of these approaches remain methodologically flawed. To remedy these flaws, this paper combines 1200 state-level polls during the 2012 presidential campaign with over 100 million statelocated political Tweets; models the polls as a function of the Twitter text using a new linear regularization feature-selection method; and shows via out-of-sample testing that when properly modeled, the Twitter-based measures track and to some degree predict opinion polls, and can be extended to unpolled states and potentially sub-state regions and sub-day timescales. An examination of the most predictive textual features reveals the topics and events associated with opinion shifts, sheds light on more general theories of partisan difference in attention and information processing, and may be of use for real-time campaign strategy. Replication Materials: The data, code, and any additional materials required to replicate all analyses in this article are available on the American Journal of Political Science Dataverse within the Harvard Dataverse Network, at: http://dx.doi.org/10.7910/DVN/RJAUNW The author would like to thank for their invaluable comments and suggestions the participants at the MPSA, APSA, and New Directions in Text conference panels and the Harvard Applied Statistics and the NYU Social Media and Political Participation workshops where drafts of this paper were presented.